ReLU Network with Bounded Width Is a Universal Approximator in View of an Approximate Identity
نویسندگان
چکیده
منابع مشابه
Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations
This article concerns the expressive power of depth in neural nets with ReLU activations and bounded width. We are particularly interested in the following questions: what is the minimal width wmin(d) so that ReLU nets of width wmin(d) (and arbitrary depth) can approximate any continuous function on the unit cube [0, 1] aribitrarily well? For ReLU nets near this minimal width, what can one say ...
متن کاملNeural Network with Unbounded Activations is Universal Approximator
Abstract This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning. The ReLU network can be analyzed by the ridgelet transform with respect to Lizorkin distributions. By showing three reconstruction formulas by using the Fourier slice the...
متن کاملOn the Use of Neural Network as a Universal Approximator
Neural network process modelling needs the use of experimental design and studies. A new neural network constructive algorithm is proposed. Moreover, the paper deals with the influence of the parameters of radial basis function neural networks and multilayer perceptrons network in process modelling. Particularly, it is shown that the neural modelling, depending on learning approach, cannot be a...
متن کاملThe Learning Dynamcis of a Universal Approximator
The learning properties of a universal approximator, a normalized committee machine with adjustable biases, are studied for on-line back-propagation learning. Within a statistical mechanics framework, numerical studies show that this model has features which do not exist in previously studied two-layer network models without adjustable biases, e.g., attractive suboptimal symmetric phases even f...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Applied Sciences
سال: 2021
ISSN: 2076-3417
DOI: 10.3390/app11010427